Record Linkage Modeling in Federal Statistical Databases

Record linkage (e.g., Fellegi and Sunter 1969, Newcombe et al. 1959) involves comparing two or more files on the same population for purposes of unduplication of records and merging files. Record linkage is used in many applications, including population size estimation at the U.S. Census Bureau (Winkler 1994, 1995, and Jaro 1989, 1995), epidemiology and medical studies (Newcombe 1988, Gill 1997), sociological studies (Belin et al. 2004), survey frame improvement, and, more recently, counterterrorism (Gomatam and Larsen 2004). See also Alvey and Jamerson (1997) and references therein. Latent class (McCutcheon 1987) and mixture models (McLachlan and Peel 2000) have been used to model the data arising from comparing records in two files (Larsen and Rubin 2001, Winkler 1988, 1994, 1995, Jaro 1989, 1995). Although successful in many applications (Alvey and Jamerson 1997), the models used in these applications have not accounted for all restrictions in the data. In particular, forcing each record on one file to have at most a single, matching record on the other file (“one-to-one matching”) has been implemented post-hoc with a one-to-one, linear-sum assignment procedure (Burkard and Derigs 1980, Jaro 1989) to choose individual links. The one-to-one assignment procedure can effectively eliminate many candidate links that have some degree of similarity, but actually are nonlinks. Experience from previous record linkage operations has been used informally to select models (Larsen and Rubin 2001) and restrict parameters (Winkler 1989, 1994). Bayesian approaches to record linkage have been suggested by Larsen (1999a, 2002, 2004, 2005), Fortini et al. (2002, 2000), and McGlinchy (2004). A procedure is described here that explicitly uses the one-to-one matching assumption and allows parameter values to vary by file block, which is a subset of the data being linked. The approach will necessarily be Bayesian, because of the relatively small sample sizes within blocks and the difficulty of calculating expectations under complex restrictions on unobserved data. This article is organized as follows. Record linkage is introduced in section 2. Bayesian record linkage algorithms are presented in section 3. Section 4 is a conclusion and discusses future work. Computational procedures are presented in appendices Appendix A and Appendix B. References are given in section 5.